Improving Artificial Neural Network Estimates of Posterior Probabilities of Speech Sounds Doctoral Thesis Proposal

Speech contains information of at least three sources the message that is being communicated, the speaker who is communicating and the environment. In this work we propose several approaches to improve the recognition of speech sounds that convey information about the message. We use phonemes which occur at the rate of few tens of milliseconds in the speech signal as basic units. Improvements in recognizing these units result in considerable performance gains in applications like automatic speech recognition (ASR) where the goal is to transcribe the message into text and automatic speaker verification that uses information in the speaker component to verify the the speaker’s claimed identity. We propose several approaches to improve phoneme posterior estimates from artificial neural networks. These include combination of information for multiple acoustic streams and different neural network training architectures. For speech recognition, especially in lowresource scenarios where the amount training data is limited (for example 1 hour of training), features extracted from better phoneme posteriors using the proposed techniques provide significant word recognition improvements. For speaker recognition these posteriors are used in a recently proposed neural network architecture to give considerable improvements over the earlier neural network based approaches. In future we would like to investigate the enhancement of phoneme posteriors for these applications in noisy environments. In a multistream speech recognition framework, we propose to use statistics derived from phoneme posteriors to determine the reliability of individual streams and how the streams can be effectively combined to derive better posteriors. We would also like to explore how these posteriors can be used for reliable voice activity detection in these environments.